{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RNN on MNIST\n",
    "\n",
    "This tutorial is also available in step-by-step notebook version on [github](https://github.com/dmlc/minpy/blob/master/examples/tutorials/rnn_mnist.ipynb). Please try it out!\n",
    "\n",
    "This tutorial is contributed by [Kerui Min](http://www.bosondata.com.cn/team), CTO of [BosonData](http://bosonnlp.com/) (links are only available in Chinese).\n",
    "\n",
    "The previous Adding Problem example demostrated that RNN can handle (sparse) inputs with long-term dependencies. In this example, we apply RNN on the MNIST handwritten digits dataset to further show its effectiveness.\n",
    "\n",
    "* You can download the data from [here](https://github.com/lryta/mnist/raw/master/mnist.dat).\n",
    "\n",
    "![MNIST Dataset](http://neuralnetworksanddeeplearning.com/images/mnist_100_digits.png)\n",
    "\n",
    "First, you can define a RNN Network as in the previous section with slight modification:\n",
    "\n",
    "1. Since this is a classification task, instead of using l2_loss, we employ softmax_loss as our loss function.\n",
    "2. We initialize 'Wh' as an identity matrix, and 'b' as a zero vector. Therefore, the signal of 'h' flows easily at the begining of the optimization.\n",
    "3. We set the parameter 'h0' to zero before each forward step, to make sure it doesn't memorize information from previous samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import mxnet\n",
    "from minpy.nn.model import ModelBase\n",
    "import minpy.nn.layers as layers\n",
    "\n",
    "class RNNNet(ModelBase):\n",
    "    def __init__(self,\n",
    "                 batch_size=100,\n",
    "                 input_size=1,\n",
    "                 hidden_size=64,\n",
    "                 num_classes=10):\n",
    "        super(RNNNet, self).__init__()\n",
    "        self.add_param(name='h0', shape=(batch_size, hidden_size))\\\n",
    "            .add_param(name='Wx', shape=(input_size, hidden_size))\\\n",
    "            .add_param(name='Wh', shape=(hidden_size, hidden_size),\n",
    "                       init_rule='constant',\n",
    "                       init_config={'value': np.identity(hidden_size)})\\\n",
    "            .add_param(name='b', shape=(hidden_size,),\n",
    "                       init_rule='constant',\n",
    "                       init_config={'value': np.zeros(hidden_size)})\\\n",
    "            .add_param(name='Wa', shape=(hidden_size, num_classes))\\\n",
    "            .add_param(name='ba', shape=(num_classes,))\n",
    "\n",
    "    def forward(self, X, mode):\n",
    "        seq_len = X.shape[1]\n",
    "        self.params['h0'][:] = 0\n",
    "        h = self.params['h0']\n",
    "        for t in xrange(seq_len):\n",
    "            h = layers.rnn_step(X[:, t, :], h, self.params['Wx'],\n",
    "                         self.params['Wh'], self.params['b'])\n",
    "        y = layers.affine(h, self.params['Wa'], self.params['ba'])\n",
    "\n",
    "        return y\n",
    "\n",
    "    def loss(self, predict, y):\n",
    "        return layers.softmax_loss(predict, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training data consists of 60000 samples, each of which is a 784-dimensional uint8 vector, representing a 28*28 grey image. Usually, people treat each image as a 784-d vector, and build classifiers based on this representation. In this case, however, we treat each 784-d vector as a sequence.\n",
    "\n",
    "Imagine that instead of reading the whole image, at each step, we are only allowed to read few pixels (a patch) of the given image to determine which class it belongs to at the end. This is much more difficult, as the final decision cannot be made with one or two patches.\n",
    "\n",
    "To make the dataset easiler to learn, we need to normalize the data before training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import joblib\n",
    "import numpy as np\n",
    "data = joblib.load(\"data/mnist.dat\")\n",
    "\n",
    "mean = np.mean(data[\"train_data\"], axis=0)\n",
    "std = np.std(data[\"train_data\"] - mean, axis=0)\n",
    "data[\"train_data\"] = (data[\"train_data\"][:] - mean)/(std+1e-5)\n",
    "data[\"test_data\"] = (data[\"test_data\"][:] - mean)/(std+1e-5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an example, we set the size of each patch to 7. Hence, the length of each sample is 112 (784/7). RNN needs to classify each sample after reading the whole 112 patches. Notice that we only use 5000 samples for training, 1000 for testing, for faster demonstration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 1 / 10000) loss: 2.817845\n",
      "(Iteration 11 / 10000) loss: 1.965365\n",
      "(Iteration 21 / 10000) loss: 1.868933\n",
      "(Iteration 31 / 10000) loss: 1.466141\n",
      "(Iteration 41 / 10000) loss: 1.434501\n",
      "(Iteration 51 / 10000) loss: 1.485497\n",
      "(Iteration 61 / 10000) loss: 1.249973\n",
      "(Iteration 71 / 10000) loss: 1.580822\n",
      "(Iteration 81 / 10000) loss: 1.350305\n",
      "(Iteration 91 / 10000) loss: 1.369664\n",
      "(Epoch 1 / 100) train acc: 0.509000; val_acc: 0.470000\n",
      "(Iteration 101 / 10000) loss: 1.350214\n",
      "(Iteration 111 / 10000) loss: 1.396372\n",
      "(Iteration 121 / 10000) loss: 1.294581\n",
      "(Iteration 131 / 10000) loss: 1.278551\n",
      "(Iteration 141 / 10000) loss: 1.132157\n",
      "(Iteration 151 / 10000) loss: 1.147118\n",
      "(Iteration 161 / 10000) loss: 0.856366\n",
      "(Iteration 171 / 10000) loss: 1.439825\n",
      "(Iteration 181 / 10000) loss: 1.113218\n",
      "(Iteration 191 / 10000) loss: 1.132181\n",
      "(Epoch 2 / 100) train acc: 0.609000; val_acc: 0.555000\n",
      "(Iteration 201 / 10000) loss: 1.147544\n",
      "(Iteration 211 / 10000) loss: 1.187811\n",
      "(Iteration 221 / 10000) loss: 1.008041\n",
      "(Iteration 231 / 10000) loss: 1.207148\n",
      "(Iteration 241 / 10000) loss: 1.075240\n",
      "(Iteration 251 / 10000) loss: 1.074992\n",
      "(Iteration 261 / 10000) loss: 0.694210\n",
      "(Iteration 271 / 10000) loss: 1.378169\n",
      "(Iteration 281 / 10000) loss: 0.997993\n",
      "(Iteration 291 / 10000) loss: 1.032932\n",
      "(Epoch 3 / 100) train acc: 0.614000; val_acc: 0.595000\n",
      "(Iteration 301 / 10000) loss: 1.046198\n",
      "(Iteration 311 / 10000) loss: 1.099993\n",
      "(Iteration 321 / 10000) loss: 0.825924\n",
      "(Iteration 331 / 10000) loss: 1.125248\n",
      "(Iteration 341 / 10000) loss: 0.977916\n",
      "(Iteration 351 / 10000) loss: 0.967498\n",
      "(Iteration 361 / 10000) loss: 0.586793\n",
      "(Iteration 371 / 10000) loss: 1.243312\n",
      "(Iteration 381 / 10000) loss: 0.900940\n",
      "(Iteration 391 / 10000) loss: 1.037108\n",
      "(Epoch 4 / 100) train acc: 0.662000; val_acc: 0.630000\n",
      "(Iteration 401 / 10000) loss: 0.989269\n",
      "(Iteration 411 / 10000) loss: 0.970790\n",
      "(Iteration 421 / 10000) loss: 0.732133\n",
      "(Iteration 431 / 10000) loss: 1.103309\n",
      "(Iteration 441 / 10000) loss: 0.793366\n",
      "(Iteration 451 / 10000) loss: 0.851175\n",
      "(Iteration 461 / 10000) loss: 0.745199\n",
      "(Iteration 471 / 10000) loss: 1.086999\n",
      "(Iteration 481 / 10000) loss: 0.754697\n",
      "(Iteration 491 / 10000) loss: 0.927628\n",
      "(Epoch 5 / 100) train acc: 0.719000; val_acc: 0.674000\n",
      "(Iteration 501 / 10000) loss: 0.899225\n",
      "(Iteration 511 / 10000) loss: 0.835000\n",
      "(Iteration 521 / 10000) loss: 0.632679\n",
      "(Iteration 531 / 10000) loss: 0.956496\n",
      "(Iteration 541 / 10000) loss: 0.699549\n",
      "(Iteration 551 / 10000) loss: 0.734873\n",
      "(Iteration 561 / 10000) loss: 0.666981\n",
      "(Iteration 571 / 10000) loss: 0.838450\n",
      "(Iteration 581 / 10000) loss: 0.668199\n",
      "(Iteration 591 / 10000) loss: 0.885664\n",
      "(Epoch 6 / 100) train acc: 0.749000; val_acc: 0.680000\n",
      "(Iteration 601 / 10000) loss: 0.853983\n",
      "(Iteration 611 / 10000) loss: 0.760786\n",
      "(Iteration 621 / 10000) loss: 0.512938\n",
      "(Iteration 631 / 10000) loss: 0.808366\n",
      "(Iteration 641 / 10000) loss: 0.572112"
     ]
    }
   ],
   "source": [
    "from minpy.nn.io import NDArrayIter\n",
    "from minpy.nn.solver import Solver\n",
    "\n",
    "BATCH = 50\n",
    "INPUT_DIM = 7\n",
    "HIDDEN_DIM = 128\n",
    "\n",
    "_, dim = data[\"train_data\"].shape\n",
    "seq_len = dim / INPUT_DIM\n",
    "\n",
    "train_iter = NDArrayIter(data[\"train_data\"][:5000].reshape(5000, seq_len, INPUT_DIM),\n",
    "                         data[\"train_label\"][:5000],\n",
    "                         batch_size=BATCH,\n",
    "                         shuffle=True)\n",
    "\n",
    "test_iter = NDArrayIter(data[\"test_data\"][:1000].reshape(1000, seq_len, INPUT_DIM),\n",
    "                        data[\"test_label\"][:1000],\n",
    "                        batch_size=BATCH,\n",
    "                        shuffle=False)\n",
    "\n",
    "model = RNNNet(batch_size=BATCH, input_size=INPUT_DIM, hidden_size=HIDDEN_DIM)\n",
    "\n",
    "solver = Solver(model,\n",
    "                train_iter,\n",
    "                test_iter,\n",
    "                num_epochs=100,\n",
    "                init_rule='xavier',\n",
    "                update_rule='rmsprop',\n",
    "                optim_config={\n",
    "                        'learning_rate': 0.0002,\n",
    "                },\n",
    "                verbose=True,\n",
    "                print_every=10)\n",
    "\n",
    "solver.init()\n",
    "solver.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A typical learning curve for this problem look like the following figure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAFzCAYAAADsTAnbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3XlclNX+B/DvDIjiCrHjRlkKLhgzIGDaohCaaVzvFcVb\nIiJmmmmomWtKZqaGZO4LLpVI3oosEhP0YsoizqikguaGLA6CglsoOvP8/jj3+TngALM/LJ/36zUv\nnGee5zxnuDfmO+d8z/eIOI4jAAAAAKGIhe4AAAAANG8IRgAAAEBQCEYAAABAUAhGAAAAQFAIRgAA\nAEBQCEYAAABAUAhGAAAAQFAIRgAAAEBQCEYAAABAUIIHI+vWrZvq5uZ21drautLPzy8zOzvbp77z\nPTw8clu3bv23u7t73jfffPOO+us7duwYLxaLVeqP1q1b/23adwEAAAD6shTy5gkJCaNnzpz55aZN\nm9719fXNWr169YdBQUEHzp8/38PBwaG05vkbNmx4b968ecu2bt060cfHJzsrK8s3MjJyi62tbfmb\nb775K39e+/bt71y4cKE7/1wkEqHmPQAAQAMlEnJvGl9f3yxfX9+sNWvWfEBExHGcqHPnzgXTpk37\nes6cOV/UPL9///7pAwcO/OOLL76Ywx+bNWvWqqysLN8//vhjIBEbGfnwww9Xl5eX25rvnQAAAIC+\nBBsZqaqqspLL5ZL58+d/xh8TiURcQEBASkZGhn9t17Rs2fKh+rFWrVo9OH78eD+lUmlhYWGhJCK6\nd+9eWzc3t6sqlUoskUjky5Ytm9ezZ89zNdsrKyuzO3DgQND/pokeGPs9AgAANFWVlZWtrl696hYU\nFHTA3t7+pkGNcRwnyKOoqMhVJBKpMjMzfdWPz549e4Wvr2+mpmvmzZv3mYuLS7FMJpOoVCpRdna2\nt5OTk0IsFisVCoUTx3GUkZHh980337x9+vRpz7S0tJeHDx++r0OHDhWFhYUda7b37bffjiUiDg88\n8MADDzzw0O/x7bffjjU0JhA0Z0RXCxcu/FShUDj7+fllchwncnZ2VowfP37HihUrPhKLxSoiIj8/\nv0w/P79M/pr+/fune3h45G7atOnd6OjoRertPfvss1eJiL799lvy8PAw63tpzmbMmEGxsbFCd6NZ\nwe/c/PA7Nz/8zs0rNzeX3n777f//LDWEYMGIvb19mYWFhbKkpMRJ/XhJSYmTi4vLdU3XtGrV6sG2\nbdsiNm/ePIk/b+PGjZPbtWt3V1PCKxGRpaXlYy8vr5MXL158XlN7REQeHh4kkUiM8bZACzY2Nvh9\nmxl+5+aH37n54XcuDP6z1BCCLe21srKqkkqlspSUlAD+mEqlEqempg729/fPqOtaCwsLpaura7FI\nJOL27NkzZvjw4b/Udq5SqbTIycnxdHV1LTZm/wEAAMA4BJ2miYqKigkLC9vp7e19wsfHJzs2NnZG\nZWWldXh4+HYiorlz535eXFzsunPnzjAior/++uuFrKwsX19f36zy8nLbmJiYqHPnzvVUrzUSHR29\nyN/fP6Nbt26XKioqbFauXDm7oKCg88SJE7cK9T4BAACgdoIGIyEhId+XlpY6LFq0KFqhUDh7eXmd\nTE5OHsJPuSgUCueCgoLO/PlKpdIiJiYm6vz58z1atGjxaNCgQYfS09P7d+nS5Rp/TkVFhU1kZOQW\nhULhbGtrW+7t7X0iPT29v7u7e54Q7xEAAADqJmidEaHJ5XKJVCqVyWQyzDOaUXx8PIWGhgrdjWYF\nv3Pzw+/c/PA7Ny+5XE5SqZRkMplUIpHIDWlL8HLw0Pzgj4X54Xdufvidmx9+540XghEAAAAQFIIR\nAAAAEBSCEQAAABAUghEAAAAQFIIRAAAAEBSCEQAAABAUghEAAAAQFIIRAAAAEBSCEQAAABAUghEA\nAIAm4tgxospKoXuhOwQjAAAATcCmTUQDBhC9/77QPdEdghEAAIBG7sABoqlTiXx8iOLiiPbvF7pH\nukEwAgAA0Ij9+SfRqFFEQ4cSpacTvf46UWQk0e3bQvdMewhGAAAAGqnr14nefJOoWzei+HgiS0ui\nLVuI7twhiooSunfaQzACAADQCN2/TzRiBNHjx0S//ELUti073qULUUxM45quQTACAADQyCiVRO+8\nQ5SbS/Trr0SdOlV/PSKicU3XIBgBAABoZObMIfr5Z6I9e4i8vJ5+XSRqXNM1CEYAAAAakY0bib78\nkig2luWL1EZ9uiY52Xz904el0B0AAABoKCoriRYvZh/e9vZEDg7s4ehY/aeDA5GrK1H79ubt34ED\nrI7ItGnsUZ+ICKK9e9l0zZkzRB06mL6P+kAwAgAAQEQZGUTh4URXrxKFhrLA5MYNorNn2c+yMiKV\n6sn5LVoQXbpE1LmzefrHL+EdMoRo9WrtruGna3r3ZtM127aZto/6QjACAADNWmUl0aJFbErDx4fo\n5EkiD4+nz1OpiMrLWWBy5QrRsGFE2dnmCUZKS9n9unVjeSIWFtpf26ULm9aZNOlJMNPQIGcEAACa\nrYwMlgD69ddEy5ezvV00BSJERGIxkZ0de/2NN9iUTU6Oefr5669EBQXVl/DqYuLEhr26BsEIAACY\nREWF0D2oXWUl0ezZbC8XGxs2GjJ7tm4jDp6e5gtG8vOJnJ2fXsKrLX665vbthrm6BsEIAAAYXW4u\n+/DculXonjxNl9GQuvTpY95gpGtXw9rgp2vi4timeuXlxumbMSAYAQAAo5s5k+jhQ7b8lOOE6wfH\nsZLpqalEa9awQmGGjIao8/RkCaz37hm3z5pcu8aCCUNNnEg0ciTR5MlsysnLi+jDD1nNEiGDEySw\nAgCAUe3fzx5TpxKtW0f0xx9EL79s+vveuMFGKs6dYytgzp5l/+Y/ZFu2JHJ3J/riC/YBrG8QwvP0\nZD/PnCHy8zOsrfrk5xNJJIa3IxIR/fADWzH03/+yx48/sqBRJCLq25fo1VfZY+BAomeeMfye2hB0\nZGTdunVT3dzcrlpbW1f6+fllZmdn+9R3voeHR27r1q3/dnd3z/vmm2/eqXnO3r17R7m7u+dZW1tX\nenp65uzfv3+o6d4BAACoe/SI5SS8+iobiejenWjDBtPe88EDorlziVxciAIDiT76iCgri40kzJxJ\n9NNPROfPsxGMU6eIZs0yPBAhIurZkyW1/vmn4W3VRaViyauGTtOoc3MjGj+eaMcOFuhcucKmb/r2\nZb+v4GCijh3Z6JZZcBwnyGPPnj2jW7Zs+WDHjh1hubm57pMmTdpka2t768aNGw6azl+/fv177du3\nv/3999+PunLlituePXtGt2vX7s4vv/zyJn/OsWPH+ltaWj5atWrVzLy8vB4LFy6MtrKyenjmzJle\nmtqUyWQSIuJkMhkHAACGW7OG40Qijjt5kj1fvZrjWrTguOvXTXO/rCyO69mT3ePTTznu/HmOe/TI\nNPfSxMOD495/37T3KCriOCKO+/ln095H3ZUrHJeUVPc5MpmM+99nqIQzMCYQLBjp169f1rRp09bw\nz1Uqlahjx46Fy5cvn6PpfH9///SPPvroC/VjM2fOXDVgwIA/+OchISEJw4cP36d+jp+fX8bkyZM3\naGoTwQgAgPHcvMlxtrYcN3Hik2O3bnGctTXHLV1q3HtVVnLcxx9znFjMcVIpx/35p3Hb19bo0Rz3\n8sumvUdGBvu0PnXKtPfRlTGDEUGmaaqqqqzkcrkkICAghT8mEom4gICAlIyMDP/armnZsmW1AaNW\nrVo9OH78eD+lUmlBRJSZmemn3iYRUVBQ0IHa2gQAAONZsoRtZ7906ZNjtrZEY8ey1RuPHxvnPseP\nE0mlrEjZ0qVEmZmswqgQ+OW9pkzSzc9nP405TdPQCJLAWlZWZq9UKi2cnJxK1I87OjreyMvLc9d0\nTVBQ0IGtW7dODA4OTvTy8jopk8mkW7dunfj48WPLsrIyeycnpxKFQuGsqU2FQuFcV39mzJhBNjY2\n1Y6FhoZSaGionu8QAKB5yc1lyarLlhE5OVV/bcoUVoY8KYnorbf0v8eDByzgWbGCJXPKZMIFIbw+\nfVg9lcJC01Vizc9ne+DU+Jgyq/j4eIqPj692rMKIhWQazWqahQsXfqpQKJz9/PwyOY4TOTs7K8aP\nH79jxYoVH4nFYlX9LdQuNjaWJMZIUwYAaKZmzmTf3KdPf/o1iYTI15do/Xr9g5Hjx9m+MRcvstGQ\n2bOJLBvAJxi/oiYnx7TBiDGW9RpC0xd0uVxOUqnUKO0LMk1jb29fZmFhoSwpKakWP5eUlDi5uLhc\n13RNq1atHmzbti2isrLSOj8/v+u1a9e6dO3aNb9du3Z3HRwcSomInJ2dFbq0CQAAhuOX8q5axZbP\najJlCtHvvxP99Zfu7X/3HZG/P1Hr1mw0ZO7chhGIELEgoX170xY/u3ataU/REAkUjFhZWVVJpVJZ\nSkpKAH9MpVKJU1NTB/v7+2fUda2FhYXS1dW1WCQScXv27BkzfPjwX/jX/P39M9TbJCI6ePBgYH1t\nAgCAfh49YqMir77KloPWJiSE1azYuFG39i9dYgW6QkNZ5VShp2VqEonY6Igpl/cao/pqQydYbBkV\nFRUTFha209vb+4SPj092bGzsjMrKSuvw8PDtRERz5879vLi42HXnzp1hRER//fXXC1lZWb6+vr5Z\n5eXltjExMVHnzp3rqV5rZPr06V+98soraTExMVFvvPHGb3v27Bkjl8slW7dunSjU+wQA0JVSyYpS\n8YW77t1j5cp79WJFu6ythe7hE5s2EeXlEe3ezT6Ya9OqFVFEBCsP/+mnbJSjPo8fE739NstB2bCh\n4YyG1OTpSZSWZrr28/OJ/v1v07XfEAj2P21ISMj3paWlDosWLYpWKBTOXl5eJ5OTk4fwUy4KhcK5\noKDg/2fglEqlRUxMTNT58+d7tGjR4tGgQYMOpaen9+/Spcs1/hx/f/+M3bt3j12wYMHSefPmLeve\nvfuFxMTE4J49e54T4j0CANRFqWTFpmpWDM3NZcmaREQdOrBdWouK2HORiOi551hg0rPnk58eHuYP\nUm7dIvrkExZkvPhi/ee/+y6byklIYPkf9fnsM6LsbKKjR4natTO8v6bi6cmCsocPa5+m0tft20R3\n7jT9kRERZ8r1SA2cXC6XSKVSmUwmQwIrgJkVFLAt2I39x7sxGTSI6PBh9u8OHaoHGPy/XV1ZAHLn\nzpOgRT14KSxk19vYEJ04QdStm/n6P3060fbtLA+k5gqa2rzxBlFpKQsy6pKZyfaQWbCAaPFig7tq\nUhkZRP37s71utAnKdJGTw6qipqezvJmGhE9glclkUolEIjekrQY66AUATVlxMfsm/8knbFVEc3T/\nPhvanzePJXfyQUdt2rdn+5/U3APl9m0WnAwbxr6dr1hh2n7z6lrKW5cpU4iGD2fBiE8tG4Dcvcum\nJXx8WDDS0PF5LDk5xg9GmkONESLs2gsAAvjkE/ZhnJkpdE+EI5OxPUdGj2Z7gNQViNSlQwf2jTks\njI1SmGsvkVmzal/KW5ehQ9l1de1XM306UUkJ0bffNtw8EXXt2hE9+6xpVtTk5xO1aEHkXGe1rMYP\nwQgAmNW5c2xDLhcX9oHcXGVlEbVpw6ZjjCEykqisjG0Fb0onTrDA57ff6l7KWxsLC5Y7Eh/Pck5q\n+uEHFlStWWPeKSdD8ZVYje3aNVa/RNzEP62b+NsDgIbm44/ZN+PPP2ff+m7eFLpHwsjKIvL2Ns7u\nsUQsv2TAADZVY2wPHhDt3MkKl/n4sG3nY2LqXspbl4gIlry7Y0f140VFRJMmEY0cqV2Ca0NiqmCk\nOSzrJUIwAgBmdOQI0S+/sDwDPhnv5Elh+ySUrCyifv2M2+a77xIdOqRfYTFNrlwh+ugjok6d2Hbz\ntrZs5OXyZaIPP9R/asnRkWjUKDZVo/pf/WyVit2jZUuizZv1b1sonp5saunGDeO22xCqr5oDghEA\nMAuOY8mq3t6sANbzz7Mlq81xqqa4mK2C8fU1brv//CcLGLZs0b8NlYpNwQwbxqZJtmxh0zIXLhAl\nJxONGGGc0ZwpU1hp95T/bW361Vfs3zt3EtnZGd6+ufFl4Y1d/Kw5VF8lQjACAGbyn/+w/UVWrGDz\n32IxkZcXkdygBYGN0/Hj7KexgxFra8MTWcPDWSCiULACZUVFRF9+SfTCC8bta//+7AN8/Xr2Af7x\nx0QzZhAFBhr3PubSrRv7/RtzqubhQ6Lr1xGMAAAYRVUV20/kjTeIXnvtyXGptHkGI1lZbClvp07G\nb5tPZE1M1P3a9HSiXbtYgHDiBNGECdpVStWHSMRGR375hY3odO/O8ogaKwsLloxszGCkoID9RDAC\nAGAEmzez/IMvvqh+XCJhQ/W3bwvTL33k5LDaIIbIyjL+qAiPT2TdvFm361QqtqRWKmW5J+bI2fj3\nv9mKovx8thleq1amv6cpGTuJla8xgpwRAAAD3blDtGQJmz6ouckZX/i4MSWxJiWxb/BXruh3vVLJ\nCn6ZKhgh0i+R9dtv2WhIbKz5lpG2bcuCpoSEJzkXjZmnJ6uK+/ixcdq79r/NTjp3rvu8pgDBCACY\n1MqVbKO36OinX+M3fTNkqubRI6LUVJYgaw7l5eznwYP6XZ+by34fpgxGdE1kvXePTaOFhLBRFXMa\nM0b/JcINjacny/Mw1mqm/HxW7KyxjxhpA8EIAJhMcTFLfvzwQ835ERYWrHy2IStqtm8nCghgS1DN\nEZDwhbp+/12/67Oy2BSIVGq8PtWkayLrF1+wei81p9FAN336sJ/GWlHTXJb1EiEYAQAT+uQTlgA5\nZ07t5xiaxJqaykYBVq0yT0DCj4ykprIpF10dP84SHU29C+2kSdolsubns9/drFlEbm6m7VNTZ2/P\nEpONlTfSXJb1EiEYAQAT4cu+L1zI9k+pjURCdP48myrQFcexXW+nTGHlw80RkNy6xT60KypYjoWu\nTJm8qs7Dg2jgwPoTWT/+mO34+/HHpu9Tc2DMJNbmUn2VCMEIAJjIxx+zD+333qv7PImEBQ+nTul+\nj7Nn2Xb0r71GNG2aeQKS8nKi119nu+jqOlVz/z4bwjdHMELERkfqSmRNTyfas4cl5LZta54+NXV9\n+hgnGFGp2NJeBCMAAHpSL/tuZVX3uT17shLg+kzVHDrE2u/fnz03R0BSXs7KmQ8erHswwu/Ua65g\npK5EVn4pr0RCNG6cefrTHHh6shENQ5erKxSsPg9yRgAA9KBe9n3UqPrPb9GC/QHXJxg5fJjtcWNt\n/eSYqQOSW7fYB/zrrxNlZLCly9oy9k699akrkVWIpbzNgbHKwvPLejEyAgCgA44j+vVXtqvr8eNs\nSa+2H3ISie4rapRKorS06hVdeaYKSB49YrktzzzDghGlku1gq62sLJawa6yderWhKZFVfSnvwIHm\n60tz4O5OZGlp+FQNX/AMwQgAgBbUg5Dhw9nqmcOHiV59Vfs2pFKW8Pr339pfc/o0mzLRFIwQmSYg\n4VfS2NoSPfcce+gyVWOu5FV1fCLrpk1Pjq1YgaW8pmJlxX7nho6M5OezFVd1JX83JZZCdwAATK+q\nqv7cDV1xHKtGungxG9UYOJDlcLz6qu6lxCUSlsOgS3Ln4cNsGqKu86dNYz8/+ID9XLHCsDLn6sEI\nERsd0TYYMdVOvdqYNInonXfYzrstW7JRq5kzsZTXVIyxooZf1muOsvwNAUZGAJq4q1fZyo8BA4ji\n41lgYghNIyGHDj2ZMtHnj2fv3ix3RJepmkOHiF56iX241kV9hOS333Tvmzq+4Nkzz7Cfr7/OVqpo\nUxreVDv1aoNPZN26FUt5zcHTkwXWKpX+bTSnZb1ECEYAmrx9+9gfRSsrorFj2T4XCxY82RFUGxzH\naoFs2mTcIITXsiULSLRNYn38mOiPP4gGDdLu/PffZ+//8mX9+0j09MjIa6+x/A9tSsObcqfe+vCJ\nrOvXP1nKa+qia81Znz5Ed+8+yfvQB4IRAGhSkpKIXnmFBQ5nz7KkxTVr2BD9yJFEKSlP51OoBx+h\noexD1N2daOpUNodtrCBEnUSifTAik7E/9rXli9QkEhHZ2bE8CUPUHBmxsWEjHdoGI0KMivAmTWJ1\nTrCU1/T4FTWGTNU0p1LwRAhGAJq0e/fYao9hw9jznj2Jvv6aqKiIaO1alkMQGMgS7r76SnPwceUK\n+1a9fz8bGUhNNW4QwpNKic6c0W4vlUOHWJEuXfZ3MUYwUl7ORnHUlxIHBrKArq7S8ObYqbc+Hh5E\nMTFEO3ZgKa+pubqygFXfYOT2bbZkvDmNjCCBFaCBksnYh17Pnvq3kZrKckT4YITXrh2rjDp5Mpvu\nWLeO7U3Ccaw+SFgYS0R96SXzDedLJGzp7Jkz9QcZhw8TvfwyyzPRlr29cUZG+FER3uuvEy1Zwmp2\n1BZs8Dv19utn2P0N9eGHwt6/uRCJDEtibW7LeokQjAA0SA8fEo0YwfILsrL0bycpieiFF9hDE5GI\nfai//DL7oG3RQrhcAk9Pln8hl9cdjDx8SHT0KFF0tG7tG2tkhM8X4fXr96Q0fG3BCL9Tr7e3YfeH\nxsPTkyg5Wb9r+WAE0zQAIKjdu9lS0OPHtVupoQnHsdUjNUdFavPMM8ImNfKjQPWtqDl+nKiyUvt8\nEZ6dHSv+ZYjy8qdHRiwt6y8Nn5Vlnp16oeHw9CS6eFG32jm8a9fYFwMXF+P3q6FCMALQwKhUrB5G\nYCD7gE5I0K+dnByWG6JtMNIQaJPEevgwSxx98UXd2jZWAmvNkREiNlWTmVl7afjjx4XNFwHz8/Rk\n/y2fO6f7tfn5bNVbc8rtEfytrlu3bqqbm9tVa2vrSj8/v8zs7Gyfus7ftWvXOE9Pz5w2bdrcd3V1\nLY6IiNh269at//+usmPHjvFisVil/mjdurUesSmAMH79lSgvjxUTGz6cLcXUR1ISS/J8+WWjds+k\nJBIWRD16VPs5hw6x1UG6llQ31jRNzZERIhY4Pn6suTS8uXfqhYahVy82NadP3khzW9ZLJHAwkpCQ\nMHrmzJlfLlmy5JOTJ0969e3b93RQUNCB0tJSB03np6WlvTJhwoS4SZMmbT537lzPvXv3jjp+/Hi/\nyMjIantStm/f/o5CoXDmH/n5+c3sf1ZozFasYImj/fsTjRnDyp7n5eneTlIS+5A0duVVU5JKWU5I\nbd8mKyvZ5nS6TtEQsWDk3j3Dir7VNjLSrVvtpeHNvVMvNAytWxM9/7z+wUhzyhchEjgYiYmJiZo0\nadLmsLCwne7u7nkbN26c3Lp167/j4uImaDo/Ozvbx83N7er777+/tmvXrvkvvfTSsUmTJm0+fvx4\ntRx1kUjEOTo63uAfDg4OpeZ5RwCGOXaMPebMYc+HDmV5BrpO1dy8yaYN3njD+H00pb592bfJ2qZq\nMjJYMKFtsTN1dnbspyGjI5oSWHm1lYbPymIfTIasioLGSd8VNXwp+OZEsGCkqqrKSi6XSwICAlL4\nYyKRiAsICEjJyMjw13RNYGDgQYVC4bx///6hHMeJSkpKnPbu3Ttq2LBhSern3bt3r62bm9vVLl26\nXAsODk48d+4c/gxAo7ByJasHwed5tGpFFBzMpmp02egtOZl9G29swUjbtkQ9etQejBw6xJbo9uql\ne9uGBiMcp3lpL48vDX/1avXjWVlsFY0l1i42O3wwost/uw8fEl2/3vyCEcH+8ygrK7NXKpUWTk5O\nJerHHR0db+Tl5blruqZv376nd+3aNW7UqFF7q6qqrB4/fmw5YsSIfWvXrn2fP8fd3T1v+/bt4Z6e\nnjkVFRU2q1atmtW/f//0s2fP9urYsWORpnZnzJhBNjY21Y6FhoZSaGioEd4pgHZyc4l+/pkoLq56\n4tqYMUTffMP+qPXtq11bSUlEXl6s+FJjI5XWvqLm8GE2RaNPYp+hwUhlJRuVqW1kRL00fGTkk+NZ\nWayQHDQ/np7s/2/Xr2v/3yK/TUNDm6aJj4+n+Pj4ascqKiqMdwOO4wR5FBUVuYpEIlVmZqav+vHZ\ns2ev8PX1zdR0TUZGhl/79u1vr1q1auaff/7Z+8CBA697enqejoiI2FrbfR49emT5/PPP/7Vw4cLo\nmq/JZDIJEXEymYwDENqECRzn6spxDx5UP/7wIcc98wzHzZ2rXTuPHnGcrS3HLVhg/D6aw5dfcpy1\nNcc9flz9+N27HGdpyXHr1+vXbmkpxxFx3A8/6Hd9QQG7/rffaj/H35/j/vWvJ8+Litg1//mPfveE\nxu3SJfa/f3Ky9tekpLBrLlwwXb+MRSaTcf/7DJVwBsYEgk3T2Nvbl1lYWChLSkqc1I+XlJQ4ubi4\nXNd0zerVqz8MCgo6MHPmzC979+595vXXX/99/fr1U+Li4ibUbIdnaWn52MvL6+TFixefN8X7ADCG\n4mI2+jFjxtO70FpZsV1XtZ2qycxkuQ2NaUmvOqmUjULUTNo9epStWNEneZWIjWiIRPqPjNTcJE+T\n11+vXhpeyJ16QXhubiznS5fChdeusZ+dO5ukSw2WYMGIlZVVlVQqlaWkpATwx1QqlTg1NXWwv79/\nhqZrOI4TWVhYVNsBQiwWq/jXNF2jVCotcnJyPF1dXYuN2X8AY/rqK1ZTZNIkza+PHs2Kn2Vn199W\nUhLLq/Cpc5F8w8XXD6mZN3L4MCsC1aOHfu1aWLD6JIYGI7XljBCxYKSigpWGJxJ2p14QnlhM9Oab\nRP/5j/bX5OcTOTuzfLHmRNDVNFFRUTFbtmyJ3LVr17jc3FyP9957b0NlZaV1eHj4diKiuXPnfh4W\nFraTPz84ODjxhx9++OfGjRsnX758+bljx4699MEHH6zx9fXNcnZ2VhARRUdHLzp48GDg5cuXn5PL\n5ZK3337724KCgs4TJ07cKtT7BKjL7dtEGzeyfWI6dNB8zquvEjk5aVdzJCmJrcLRtQ5HQ9GhA1sS\nWTMYOXTI8A36DNmfht+xt66REb40PL+Lr9A79YLwxoxhdWbOntXu/Oa4rJdI4GAkJCTk+1WrVs1a\ntGhRtJeX18mcnBzP5OTkIfxSXIVC4VxQUPD/g1Vjx47d/dVXX01fu3bt+3369PkzJCTkew8Pj9wf\nf/xxJH9LfSvbAAAgAElEQVRORUWFTWRk5JaePXueGzZsWNK9e/fapqen93d3d9ejUgOA6W3eTPTg\nAdH06bWfY2FBNGoU0fffs1UytSkoYH/4GusUDa9mEmtFBQtO9J2i4RlS+EybaRr10vD8Tr1Cb44H\nwgoKYgG2tsvzm+OyXiISLoG1ITyQwApCe/CA41xcOC4iov5zjx5liW1HjtR+zsaNHGdhwXHl5cbr\noxC++ILj2rblOKWSPf/5Z/beL10yrN1hwzhu+HD9rl21iuPatav/vPXrWaJtejrr86FD+t0Pmo7w\ncI574QWOU6nqP7dbN46bNcv0fTKGJpHACgBE333Hlv3NmlX/uf7+LKmtrqmapCRWvbXGSvVGRyJh\n1VIvXmTPDx9mQ9fPPmtYu4aOjNQ1KsJ7/XWWaLt8OXbqBWb0aFaD5tSpus9TqdjoJqZpAMBsVCpW\n5Oytt4jcNVbWqU4sJgoJIdq7l33Y1fTgAVFqauOfoiFiwQjRk6maw4dZ1VVD8kWIDAtG6ip4po4v\nDb9vH3bqBWbQIJavVF/Ol0LBatk0x2kaBCMAAuE3xPvoI+2vGTOGqLSUfTjX9N//su3Km0Iw8swz\nbFmkXE5UVsb25zE0X4TIPCMjRGx0hAjJq8C0aEH0r3/VvzyfX9aLYAQAzEZ9QzxtSaXsm7emZLik\nJPZHrKnsgSKRsGAkLY09N1YwcutW3UnAtdF2ZIQIwQg8bcwYFmxkZtZ+Tn4++4lgBAAM9uBB/cXJ\nam6Ipy2RiP1R++GH6rvPchwLRt54w/CpjIZCKmXBSGoqW+prjCJQdnYsELl9W/drdRkZCQxk/zu9\n+abu94GmacAAVienrlU1+flsWq+2Jf5NGbZuAjCi6GiixYvZEk8HhycPR8fq/969u/qGeLoYPZro\ns8/Y8lH+wy4vjxVFawpTNDyJhC3p3bOHDXEbg/r+NNoGFjxdRkbatiWqsY0HNHMWFizn6/vvib78\nUnMdIH5Zb1P5QqELBCMARrJ4MdGSJUQzZ7JVH6Wl7HHjBlFREcukv3GDfRByHCv/rs+Gb717s6mY\nPXueBCNJSaxiozGmMhoKPom1vNx470s9GHlexw0idBkZAdBkzBhWbfmPP1ghw5ry85vnFA0RghEA\no+ADkWXLiObOrftcpZLo7l39l9/yUzUrVrA9XKytWTAyaBBR69b6tdkQOTqyMuqFhaYJRnShUrFR\nGm1HRgA08fVlwcaePbUHIwMGmL1bDQJyRqBJ4jhWinvcOLZmv+ama8akSyBC9GSPFEOMHs3qcPz2\nG8t/OHq0aU3R8KRSNgrk7Gyc9vQNRu7cYQEJRkbAEOo5X48ePf16cx4ZQTACTcrffxPFxbFCU35+\nbDj08WM2dWIKugYixtK9O5GXF/uGdfAge49NMRiJiTFu7oW1NRs90jUY0WaTPABtjB7NlqsfOlT9\n+O3bLOhFMALQiF28yAKOTp2IJk5km8r9+is7/vXXbAQhOdm49xQqEOGNGcPeY0ICK67VFP+IPfcc\nkaencdvUp9aINpvkAWjjxRfZl4maBdCa87JeIgQj0IgplezDeOhQohdeINqxgygigpVd/u03NlJg\nYUE0ciTRyy8TRUVprlyqD6EDESKWmf/gAduevCmOipiKnR37ZqoLbTbJA9AGP1Xz009EDx8+Oc4H\nI82xFDwRghFoxBYtIho+nH2wbN/OEh1XrmRFwdSJRESxsSxvZNMmw+/bEAIRIlah1N+f/RvBiPYM\nGRnBNA0Yw+jRbFrmwIEnx65dY5VaXVyE65eQEIxAo/T330QbNhDNmMG2aR8/nuUD1MbLi2jCBBbA\n8B8s+mgogQgvIoIVA9Olimtzp08wUl7OlmFjnxkwhp49ifr0qT5Vk5/P/lvWZ7l/U9BM3zY0dgkJ\nbKnl++9rf83SpaxqaXS0fvdcurRhBSJELMC6epUVWQPt6DsyYmvbfD8owPjGjCH6+Wei+/fZ8/z8\n5jtFQ4RgBBqp9euJhgx5ekqmLs7ORPPnE61bp/tS33XriBYuJPr004YTiBCxKSh8QOpG35ER5IuA\nMY0ezUZ4k5LY8+a8rJcIwQg0QtnZRCdOEE2Zovu1M2awodBZs7S/Jj6eaNo0lgA7f77u94SGRd+R\nEeSLgDF160bk4/Nkrxq+FHxzhWAEGp3169l/tEOH6n5tq1YsyTUpqXryWG3272eF08aNY9c1xz0j\nmho7O1a5trJS+2swMgKmMGYM+1tUWkp0/TqCEYBG49YtlvT17ruaN5rSBr/U98MP617qm55O9M9/\nsp1wt27FdEhToU8V1vJyjIyA8Y0axZb3rl3LniNnBMAI7t0jOnvWtPfYsYPVF4mI0L8NbZb65uSw\n5bI+Piz4QYJo06FPMMInsAIYU+fObC8aPhjByAiAge7dIwoIYHuJ3LtnmnuoVGw576hRbBM1Q6gv\n9eULWvEuXyYKCmI77+7bV/eSYWh8MDICDcmYMU/KDXTuLGxfhIRgBAz28CGbzsjJYf8+fNg090lJ\nYeXd9Ulc1YRf6rtkyZNjCgVRYCCrJ5GcTNShg3HuBQ0HRkagIfnXv9gUsLMzy2lrrhCMgEGUSpbc\nmZbGErG6dTP+HjC89evZPiXGKvBVc6lvRQUbEXnwgOj33w0ffYGGqUMHlm+kbUn4R4/YaB+CETAF\nJyeiQYPYPkzNGWbCQW8cRzR1Ktsb5YcfiF57jX2Ya7NKRVfXrhH98gsLSIy5omXGDKLNm4mmT2fF\nhwoLiY4cYaXWoWkSi9mUi7YjI9ixF0xt+3ZWc6Q5w8gI6G3hQpYAunUrUXAwOzZkCNGlS2w6xZg2\nbyZq04bo3/82brv8Ut/ffyc6dYptsNerl3HvAQ2PLrVGsEkemFqnTmwn3+YMwQjoZfVqos8+Yx/k\n4eFPjr/2GtvsyZijI1VVRFu2EIWFEbVta7x2eSNHskTWpCQiX1/jtw8Njy7BCDbJAzA9BCOgs127\nWDXSjz9+upJp27ZsqZoxg5EffyS6cYPovfeM16Y6kYglsb7yimnah4YHIyMADQuCEdDJL7+wJbET\nJ7IN4zQJCiI6dIiNaBjD+vVEr77KdroEMAaMjAA0LIIGI+vWrZvq5uZ21drautLPzy8zOzvbp67z\nd+3aNc7T0zOnTZs2911dXYsjIiK23bp1q9qfiL17945yd3fPs7a2rvT09MzZv3+/HkXDQZMjR4hC\nQlh+yMaNtSeSBgWxZNBjxwy/559/Ev3xh/GW8wIQ6T4y0rIl6s0AmJJgwUhCQsLomTNnfrlkyZJP\nTp486dW3b9/TQUFBB0pLSx00nZ+WlvbKhAkT4iZNmrT53LlzPffu3Tvq+PHj/SIjI7fw56Snp/cf\nO3bs7sjIyC2nTp16MTg4ODE4ODjx7NmzSEk0UEYG0fDhRC+9RPTdd3WXYu/bly2bNcYS3w0bWFt8\ngiyAMegajGBUBMDEOI4T5NGvX7+sadOmreGfq1QqUceOHQuXL18+R9P5K1eunNWtW7eL6sfWrFkz\nrVOnTgX885CQkIThw4fvUz/Hz88vY/LkyRs0tSmTySRExMlkMg6elpvLcdHRHNenD8cRcVy/fhx3\n5452144bx3F9+xp2/9u3Oa5tW45buNCwdgBq2ryZ40Qijnv8uP5zp0/nuJ49Td8ngMZGJpNx//sM\nlXAGxgSCjIxUVVVZyeVySUBAQAp/TCQScQEBASkZGRn+mq4JDAw8qFAonPfv3z+U4zhRSUmJ0969\ne0cNGzYsiT8nMzPTT71NIqKgoKADtbUJT8vLI/r0U1ZczMODaMUKoj59iBIT2TRNu3batTNkCNHp\n02wnSn19+y3bWXXSJP3bANDEzo7Vyam5FYAmGBkBMD1Bip6VlZXZK5VKCycnpxL1446Ojjfy8vLc\nNV3Tt2/f07t27Ro3atSovVVVVVaPHz+2HDFixL61a9e+z5+jUCicNbWpUCic6+rPjBkzyMbGptqx\n0NBQCg0N1fm9NUaXLhHt3k20dy/L0WjblmjECBaUBAXpV6I4MJDllPz+O1uSqyuOY1M0I0awNfgA\nxqReEt7evu5zUQoegCg+Pp7i4+OrHauoqDBa+42mAmtmZqZfeHj49iVLlnwSFBR0oLi42HX27Nkr\nJ0+evHHr1q0TDWk7NjaWJBKJsbraqFRUEL34Ivu3oQGIOnt7Im9vljeiTzBy9CjRmTOsngmAsemy\nP015OdvmAKA50/QFXS6Xk1QqNUr7ggQj9vb2ZRYWFsqSkhIn9eMlJSVOLi4uGgf2V69e/WFQUNCB\nmTNnfklE1Lt37zNt2rS5P3DgwD8+++yz+U5OTiXOzs4KXdoEtuLl3j2iCxeIXnjBuG0HBbHRDaWy\n7oRXTdatYxUJBw0ybp8AiJ6MhmizP82tW0Q+da7zAwBDCZIzYmVlVSWVSmUpKSkB/DGVSiVOTU0d\n7O/vn6HpGo7jRBYWFkr1Y2KxWMW/RkTk7++fod4mEdHBgwcDa2sT2AZ3rq5Ezz9v/LaHDGHfPOVy\n3a776y82ZTRtGttHBMDY+BwQbUdGME0DYFqC/amPioqK2bJlS+SuXbvG5ebmerz33nsbKisrrcPD\nw7cTEc2dO/fzsLCwnfz5wcHBiT/88MM/N27cOPny5cvPHTt27KUPPvhgja+vb5azs7OCiGj69Olf\nJScnD4mJiYnKy8tzX7x48WK5XC55//331wr1Phu6tDRWedSYm8/xfH3ZDqm6LvFdupTtZDnRoMk3\ngNpZWbFk7PqCEY5jIyNIYAUwLcFyRkJCQr4vLS11WLRoUbRCoXD28vI6mZycPMTBwaGUiCWjFhQU\ndObPHzt27O7bt293WLt27fszZ8780sbGpmLw4MGpX3zxxRz+HH9//4zdu3ePXbBgwdJ58+Yt6969\n+4XExMTgnj17nhPiPTZ09+4RyWSsoqopWFoSDR7MSsMvXKjdNX/9xVbRxMYanrcCUBdtao1UVrJK\nwhgZATAtEcdxQvdBMHK5XCKVSmUymaxZJrD+/jvL6zh3ji3jNYUtW9ieMmVlRDUWLGkUFkZ08CDR\n5csIRsC0vL2JJBK2I3RtiorYaq7ffiMailrOANXwCawymUwqkUh0nJCvDjPyzVhaGpGjI5G7xsXU\nxhEUxBJYU1PrP5cfFZk7F4EImJ42IyP8vjQYGQEwLQQjzVhaGtHLL5smX4TXpQsbddEmb4TPFYmM\nNF1/AHjaBCN8UTTkjACYFoKRZqqykuj4cZa8ampBQSxvpK4ZQYyKgLlhZASg4dA6GHFzc7u6ZMmS\nT65du9bFlB0C88jMJHr0iI2MmNqQIUQFBUS5ubWfg1ERMDddRkYQjACYltbByIwZM2J//PHHkc8+\n++yVwMDAg/Hx8aEPHz5sacrOgemkpbGh5969TX+vl19mox0HDmh+HaMiIAQ+GKlrxO7WLbYE2LLR\n1KoGaJx0CkZOnz7dNzs728fDwyN32rRpXzs7OyumTp26TiaTGaceLJjNkSNEAweap6iYtTULSGrL\nG8GoCAjBzo4t271/v/ZzUPAMwDx0/iiSSCTyNWvWfHD9+nWXTz75ZMm2bdsifHx8sl988cVT27Zt\ni+CroULD9fAhUUaGeaZoeEOGsACosrL6cYyKgFC02Z8GBc8AzEPnYKSqqsoqISFh9PDhw3+ZNWvW\nKm9v7xNbt26d+M9//vOH+fPnfzZ27NjdpugoGE92NtGDB+ZJXuUFBbF7HjlS/ThGRUAo2uxPg5ER\nAPPQeiZUJpNJt2/fHh4fHx9qYWGhHDdu3K7Y2NgZ7u7uefw5//jHP37y9vY+YZqugrEcOULUvv2T\n3XrNwcODqHNnNlUTFMSOodoqCAkjIwANh9bBiI+PT3ZgYODBjRs3Tn7rrbd+trKyqqp5zrPPPntl\nzJgxe4zbRTC2tDSiAQN030nXECLRkyW+PIyKgJC0CUbKy1mtHAAwLa2DkcuXLz/n5uZ2ta5z2rRp\nc3/Hjh3jDe0UmM6jR0THjmm/V4wxDRlCtHUr0bVrLG8FoyIgpLZtiVq0qD8YwcgIgOlpHYzcuHHD\nsaSkxMnX1zdL/XhmZqafpaXlY0zPNA4nT7LVA+bMF+ENHsxGYw4cIDp6FKMiICyRqP5aI7duIWcE\nwBy0TmCdOnXquqKioo41jxcVFXWcOnXqOuN2C0wlLY2odWsiqQCLsW1siHx9iTZtwgoaaBjqCkZU\nKqKKCoyMAJiD1sFIbm6uh5eX18max728vE6ePXu2l3G7BaaSlkbUvz8bnhbCkCFEMhlGRaBhqCsY\nuXOHBSQYGQEwPa2DkZYtWz68fv26S83jCoXC2dLS8rFxuwWmoFSy6REhpmh4/DbsGBWBhqCuYASb\n5AGYj9bByOuvv/77vHnzllVUVNjwx8rLy23nzp37eWBg4EHTdA+MKSeH6PZtYYMRb2+i1FSiKVOE\n6wMAr65gBJvkAZiP1gmsK1eunP3KK6+kde3aNV8ikcg5jhOdOnXqRWdnZ8U333zzjik7CcaRlkbU\nsiWRj4+w/Rg0SNj7A/C0GRlBMAJgeloHI506dSo8ffp03927d489derUi9bW1pUTJkyICw0NjW/R\nosUjU3YSjCMtjcjPD9MjADxtRkYwTQNgejrtRdm2bdt7kyZN2myqzoD2FAq2RLdbN+3OV6mI/viD\naOpU0/YLoDGxs2OJqo8ePZ3UXV7ONpJs106YvgE0JzoFIxzHiXJzcz2uXbvWpaqqykr9tREjRuwz\nbtegLuPGEZ04wfJAOnWq//xz59g3QCHzRQAaGn5/mps3iZydq7/G1xgxx87WAM2dThVY//GPf/z0\n559/9tH0ukqlwn+yZlJaypJAiYjGjyf6/ff6/2CmpbFvfn5+Ju8eQKOhXhK+ZjCCTfIAzEfrAGL6\n9Olfubm5Xb1x44ZjmzZt7p85c6b3kSNHXvb29j7x3//+91UT9hFq+Okn9vO771hQEhtb/zVHjrDE\n1datTds3gMakrv1pUAoewHy0DkYyMjL8P/3004X29vZlYrFYZWFhoRwwYMDR5cuXfzx9+vSvTNlJ\nqG7vXqLXXiMaM4YoKorV7Dh9uvbzOY6NjGCKBqC6uoIRlIIHMB+tgxGlUmnRtm3be0RE9vb2ZcXF\nxa5ERF26dLmWl5fnbqoOQnWlpUSHDhGNGsWeL1tG1KMH0b//TVRZqfmaCxeISkqIXn7ZfP0EaAz4\nYAMjIwDC0joY6dWr19mcnBxPIqJ+/fodX7FixUfHjh17KTo6etFzzz132XRdBHX8FM3Ikexny5ZE\nu3cTXbzIRkg0OXKEbVD30kvm6SNAY2FpyfZMwsgIgLC0DkYWLFiwlE9SjY6OXnTlypVnBw4c+Mf+\n/fuHrlmz5gPTdRHU8VM0Dg5PjvXuTfTFF0RffcWSWWtKSyOSSLBEEUCT2mqNYGQEwHy0Xk0zZMiQ\nZP7fL7zwwl95eXnuN2/etLO1tS0Xi8Uq03QP1PFTNOvXP/3atGlEv/1GFBZG9OefT5Ys8vkiY8aY\nt68AjUVtwQhGRgDMR6uRkaqqKitLS8vHZ86c6a1+3M7O7iYCEfOpOUWjTiwm2r6dFW+KjGRBCBHR\n1atEhYXIFwGojaZg5NEjonv3EIwAmItWwYiVlVVV586dC5RKpYWxO7Bu3bqpbm5uV62trSv9/Pwy\ns7Oza905Zfz48TvEYrGq5qN3795n+HN27NgxvubrrVu3/tvY/RaCpikada6uRFu2ECUmEsXFsWNp\naUQiEdHAgebrJ0BjoikYwY69AOaldc7I/PnzP5s3b96ymzdv2hnr5gkJCaNnzpz55ZIlSz45efKk\nV9++fU8HBQUdKC0t1fhxu2bNmg8UCoUz/ygoKOj8zDPP3AoJCfle/bz27dvfUT8vPz+/q7H6LJSa\nq2hq849/EEVEEE2fTvTXXywY6duXJekBwNPqCkYwMgJgHlrnjKxbt27qxYsXn3d1dS3u2rVrfps2\nbe7zr4lEIk4ul0t0vXlMTEzUpEmTNoeFhe0kItq4cePkpKSkYXFxcRPmzJnzRc3z27dvf6d9+/Z3\n+OeJiYnB5eXltuHh4dvVzxOJRJyjo+MNXfvTkNU1RVNTbCwLQt5+m+jGDaIRI0zbN4DGTFMwgk3y\nAMxL62Dkrbfe+rm210QiEafrjauqqqzkcrlk/vz5n6m3ExAQkJKRkeGvTRvbtm2LCAwMPNi5c+cC\n9eP37t1r6+bmdlWlUoklEol82bJl83r27HlO1z42JPVN0ahr25bo22/ZUl6lEsXOAOpib8+CEY5j\nU5pEGBkBMDetg5HFixcvNuaNy8rK7JVKpYWTk1OJ+nFHR8cb2hRRKy4udk1OTh4SHx8fqn7c3d09\nb/v27eGenp45FRUVNqtWrZrVv3//9LNnz/bq2LFjkaa2ZsyYQTY15jFCQ0MpNDRU0+lmV9cqmtr4\n+hJ98gnR0qVIXgWoi50dC9pv334ynYmcEYDq4uPjKT4+vtqxiooKo7Wv0669DcnOnTvDbG1ty4OD\ngxPVj/v5+WX6+fll8s/79++f7uHhkbtp06Z3o6OjF2lqKzY2liQSnWeZzEaXKRp1CxawlTX8Ml8A\neJp6SXg+GLl1ixUUtLYWrl8ADYmmL+hyuZykUqlR2tc6gVXTKhb+YWFhodT1xvb29mUWFhbKkpIS\nJ/XjJSUlTi4uLtfrupbjOFFcXNyEd9555xtLS8vHdZ1raWn52MvL6+TFixef17WPDYUuUzTqRKKn\ndyIFgOo07U+DgmcA5qX1yMiPP/5Y7Xv5o0ePWpw6derFnTt3hukzhWNlZVUllUplKSkpASNGjNhH\nRKRSqcSpqamDP/jggzV1XZuWlvbKpUuXukVERGyr7z5KpdIiJyfH88033/xV1z42BKWlRIcPE61b\nJ3RPAJomTcEICp4BmJfWwUjN6RAiolGjRu3t1avX2YSEhNETJ07cquvNo6KiYsLCwnZ6e3uf8PHx\nyY6NjZ1RWVlpza+OmTt37ufFxcWuO3fuDFO/btu2bRF+fn6ZmpJSo6OjF/n7+2d069btUkVFhc3K\nlStnFxQUdNanfw3BTz+xxLp//EPongA0TRgZARCewTkjvr6+WZGRkVv0uTYkJOT70tJSh0WLFkUr\nFApnLy+vk8nJyUMcHBxKiYj4WiLq19y+fbvDjz/+OLK2/XAqKipsIiMjtygUCmdbW9tyb2/vE+np\n6f3d3d3z9Omj0PgpGkdHoXsC0DS1bk3UqhVGRgCEZFAw8vfff7f++uuvp9W2SkUbU6dOXTd16lSN\nkxDbt28Pr3msQ4cOt+/fv9+mtvZiYmKiYmJiovTtT0OCKRoA86hZa6S8nKhbN+H6A9DcaB2M2Nra\nlqs/5zhOdPfu3XatW7f++9tvv33b+F0DTNEAmEfNYOTWLSKfWjemAABj0zoYWb169Yfqz8ViscrB\nwaHU19c365lnnrll/K4BpmgAzEPTyAimaQDMR+tgZPz48TtM2A+oAVM0AOajHoxwHBJYAcxN6zoj\ncXFxE/bu3fvUNm179+4dVXO1CxgOUzQA5qMejFRWEj18iJERAHPSOhj5/PPP52rafM7BwaF02bJl\n84zbLcAUDYD52NkRlZWxf6MUPID5aR2MFBQUdO7atWt+zeNdu3bNz8/P72rcbjVv/BTNqKfGoQDA\nFPjN8oie7NiLkREA89E6GHF0dLxx+vTpvjWP5+TkeNrZ2d3UdA3oB1M0AOZlZ0f0999EDx5gZARA\nCFoHI6GhofEffPDBmkOHDg1SKpUWSqXSgi/dPmbMmD2m7GRzgykaAPNSr8KKkREA89N6NU10dPSi\nq1evugUEBKTwG+OpVCpxWFjYTuSMGM/Jk0SpqUTb6t11BwCMRT0Y4UdGEIwAmI/WwUjLli0fJiQk\njP70008Xnjp16kVra+vKPn36/Onm5nbVhP1rdubMIereneidd4TuCUDzUXNkpF07IkuDN8sAAG3p\n/J9b9+7dL3Tv3v2CKTrT3P3+O9HBgyxnBH8IAcyn5sgIRkUAzEvrnJGRI0f+uHLlytk1j69YseKj\nUaNG7TVut5oflYroo4+IXnqJ6K23hO4NQPNiY0MkFj8ZGUHyKoB5aR2M/PHHHwOHDh26v+bxoUOH\n7k9LS3vFuN1qfr77juj0aaKVK4lEIqF7A9C8iMVsNAQjIwDC0DoYuXfvXtsWLVo8qnnc0tLy8Z07\nd9obt1vNy4MHRAsWEI0cSeTvL3RvAJonvgorSsEDmJ/WwUjv3r3P7NmzZ0zN4wkJCaN79ux5zrjd\nal7WriUqKiL6/HOhewLQfPHByK1bGBkBMDet0yQXLVoUPXLkyB8vXbrUbfDgwalERCkpKQHx8fGh\nmvasAe3cukX02WdEkyaxVTQAIAyMjAAIR+tgZPjw4b8kJiYGf/bZZ/N/+OGHf1pbW1d6enrmpKam\nDkYFVv19/jnRo0dEn3widE8Amjc7O6Lz5zEyAiAEradpiIiGDRuWlJ6e3v/+/fttLl++/Nzo0aMT\nZs+evbJv376nTdXBpuzqVaI1a9gqGicnoXsD0LzZ27PN8ioqMDICYG46BSNERGlpaa+MGzdul6ur\na/GqVatmDRo06FBmZqafKTrX1C1cyL6BRUUJ3RMAsLMjys9ny+wxMgJgXlpN01y/ft1lx44d4+Pi\n4iaUlpY6jBkzZs+DBw9aJSYmBvfq1eusqTvZFJ08yZbzbthA1Lat0L0BADs7NmVKhJERAHOrd2Tk\nzTff/LVLly7XkpKShi1YsGDp9evXXTZu3DhZJBJxIpGIM0cnmyK+7HtEhNA9AQCiJ1VYiTAyAmBu\n9Y6M/Pbbb2/4+PhkR0dHLxo0aNAhc3SqqUPZd4CGB8EIgHDqHRk5duzYS3369PkzODg48YUXXvjr\n888/n3v9+nUXc3SuKULZd4CGST0YwTQNgHnVG4z4+/tnbN26dWJxcbHr3LlzP//111/f7Nq1a75S\nqbT4/fffX7979247c3S0qeDLvq9YgbLvAA0JH4yIxWzXXgAwH61X07Rt2/behAkT4o4dO/bSn3/+\n2SDysTEAAB73SURBVGfWrFmrli9f/rGDg0Pp8OHDfzFlJ5sK9bLv/fsL3RsAUMcHI7a2LCABAPPR\n6z+5Hj16nF+xYsVHhYWFnfbs2TMGiazaWbWKqLiYaNkyoXsCADW1bEnUpg3yRQCEYFD8b2lp+Tg4\nODhx3759I4zVoabq6lVW9v3DD4l69BC6NwCgiZ0d8kUAhCDoYOS6deumurm5XbW2tq708/PLzM7O\n9qnt3PHjx+8Qi8Wqmo/evXufUT9v7969o9zd3fP4cvX79+8favp3Ur8ZM9gfuoULhe4JANTGzg4j\nIwBCECwYSUhIGD1z5swvlyxZ8snJkye9+vbtezooKOhAaWmpg6bz16xZ84FCoXDmHwUFBZ2feeaZ\nWyEhId/z56Snp/cfO3bs7sjIyC2nTp16MTg4ODE4ODjx7Nmzvcz3zp72229EP/9MFBODxDiAhqxP\nH6Jegv61AGimOI4T5NGvX7+sadOmreGfq1QqUceOHQuXL18+R5vrf/rpp2CxWKy8du1aZ/5YSEhI\nwvDhw/epn+fn55cxefLkDZrakMlkEiLiZDIZZyqVlRzXrRvHDRrEcSqVyW4DAABgVjKZjPvfZ6iE\nMzAmEGRkpKqqykoul0sCAgJS+GMikYgLCAhIycjI8NemjW3btkUEBgYe7Ny5cwF/LDMz00+9TSKi\noKCgA9q2aQqrVrH9LtauxVJeAAAATQSp/1lWVmavVCotnJycStSPOzo63sjLy3Ov7/ri4mLX5OTk\nIfHx8aHqxxUKhbOmNhUKhXNd7c2YMYNsbGyqHQsNDaXQ0NBartCOetKqh4dBTQEAAAgmPj6e4uPj\nqx2rqKgwWvuNshj5zp07w2xtbcuDg4MTjdFebGwsSSQSYzRVDZJWAQCgKdD0BV0ul5NUKjVK+4IE\nI/b29mUWFhbKkpISJ/XjJSUlTi4uLtfrupbjOFFcXNyEd9555xtLS8vH6q85Ozsr9GnTFPik1YQE\nJK0CAADURZCcESsrqyqpVCpLSUkJ4I+pVCpxamrqYH9//4y6rk1LS3vl0qVL3SIiIrbVfM3f3z9D\nvU0iooMHDwbW16axPXhA9MEHRIMGEY0aZc47AwAAND6CTdNERUXFhIWF7fT29j7h4+OTHRsbO6Oy\nstI6PDx8OxHR3LlzPy8uLnbduXNnmPp127Zti/Dz88vs2bPnuZptTp8+/atXXnklLSYmJuqNN974\nbc+ePWPkcrlk69atE831voieJK3+8guSVgEAAOojWDASEhLyfWlpqcOiRYuiFQqFs5eX18nk5OQh\nDg4OpUQsGbWgoKCz+jW3b9/u8OOPP45cs2bNB5ra9Pf3z9i9e/fYBQsWLJ03b96y7t27X0hMTAzW\nFLiYCpJWAQAAdCPiuOa7rYxcLpdIpVKZTCYzWgJrcDDRiRNEubnIFQEAgKaLT2CVyWRSiUQiN6St\nRrmapqFC0ioAAIDusFG2kSBpFQAAQD8YGTGSFSuQtAoAAKAPjIwYwfnzLGl11iwkrQIAAOgKwYiB\nOI7o3XeJOncmWrRI6N4AAAA0PpimMdD27URpaUQHDxJZWwvdGwAAgMYHIyMGKClhUzPjxhEFBNR/\nPgAAADwNwYgBPvyQSCwm+vJLoXsCAADQeGGaRk/JyUTx8UQ7dxLZ2wvdGwAAgMYLIyN6uH+f6L33\niAYPJnrnHaF7AwAA0LhhZEQPixcTKRQsaRU1RQAAAAyDYERHJ08SrV5N9OmnRM8/L3RvAAAAGj9M\n0+hAqSSKjGSFzWbNEro3AAAATQNGRnTw9ddEcjlRejpRixZC9wYAAKBpwMiIlq5dI1qwgGjKFCI/\nP6F7AwAA0HQgGNECx7EgxMaGaNkyoXsDAADQtGCaRgv/+Q9RUhLRTz8RtW8vdG8AAACaFoyMaGHp\nUqJhw4iCg4XuCQAAQNODYEQLV64Qvfqq0L0AAABomhCM1OPOHaK7d4k6dRK6JwAAAE0TgpF6FBWx\nnx07CtsPAACApgrBSD0KC9lPBCMAAACmgWCkHvzIiKursP0AAABoqhCM1KOoiMjenqhVK6F7AgAA\n0DQhGKlHYSGSVwEAAEwJwUg9ioqQLwIAAGBKCEbqgZERAAAA00IwUg+MjAAAAJiW4MHIunXrprq5\nuV21trau9PPzy8zOzvap6/yHDx+2nD9//mdubm5XW7Vq9eDZZ5+9sn379nD+9R07dowXi8Uq9Ufr\n1q3/1qdvDx8S3biBYAQAAMCUBN0oLyEhYfTMmTO/3LRp07u+vr5Zq1ev/jAoKOjA+fPnezg4OJRq\nuiYkJOT70tJSh7i4uAnPP//8xevXr7solUoL9XPat29/58KFC9355yKRiNOnf9evs5+YpgEAADAd\nQYORmJiYqEmTJm0OCwvbSUS0cePGyUlJScPi4uImzJkz54ua5ycnJw85cuTIy1euXHnWxsamgoio\nS5cu12qeJxKJOEdHxxuG9g/VVwEAAExPsGmaqqoqK7lcLgkICEjhj4lEIi4gICAlIyPDX9M1+/bt\nG+Ht7X1i+fLlH3fq1KmwR48e52fPnr3ywYMH1aqA3Lt3r62bm9vVLl26XAsODk48d+5cT336yFdf\nxcgIAACA6Qg2MlJWVmavVCotnJycStSPOzo63sjLy3PXdM3ly5efO3r06ABra+vKxMTE4NLSUocp\nU6asv3nzpl1cXNwEIiJ3d/e87du3h3t6euZUVFTYrFq1alb//v3Tz54926tjx45FmtqdMWMG2djY\nVDsWGhpK16+HUps2RO3bG+lNAwAANELx8fEUHx9f7VhFRYXR2hd0mkZXKpVKLBaLVd99992/27Vr\nd5eITfX861//+s+GDRvea9my5UM/P79MPz+/TP6a/v37p3t4eORu2rTp3ejo6EWa2o2NjSWJRPLU\n8agoNioiEpnsLQEAADR4oaGhFBoaWu2YXC4nqVRqlPYFm6axt7cvs7CwUJaUlDipHy8pKXFycXG5\nrukaFxeX666ursV8IELERkI4jhMVFhZqnEyxtLR87OXldfLixYvP69pHLOsFAAAwPcGCESsrqyqp\nVCpLSUkJ4I+pVCpxamrqYH9//wxN1wwYMOBocXGx6/3799vwxy5cuNBdLBarOnXqVKjpGqVSaZGT\nk+Pp6uparGsfCwsRjAAAAJiaoHVGoqKiYrZs2RK5a9eucbm5uR7vvffehsrKSuvw8PDtRERz5879\nnF9pQ0Q0duzY3XZ2djfDw8O35+bmehw5cuTl2bNnr4yIiNjWsmXLh0RE0dHRiw4ePBh4+fLl5+Ry\nueTtt9/+tqCgoPPEiRO36tq/oiIkrwIAAJiaoDkjfM2QRYsWRSsUCmcvL6+TycnJQ/gaIwqFwrmg\noKAzf36bNm3uHzx4MHDatGlfe3t7n7Czs7s5evTohKVLly7gz6moqLCJjIzcolAonG1tbcu9vb1P\npKen93d3d8/TpW8qFVFxMUZGAAAATE3EcXrVA2sS5HK5RCqVymQy2VMJrCUlRM7ORImJRG+9JVAH\nAQAAGig+gVUmk0klEonckLYELwffUKHgGQAAgHkgGKkFCp4BAACYB4KRWhQVEVlaEjk6Ct0TAACA\npg3BSC0KC4lcXIjE+A0BAACYFD5qa4FlvQAAAOaBYKQWqL4KAABgHghGalFYiJERAAAAc0AwUguM\njAAAAJgHghEN7twhunsXIyMAAADmgGBEAxQ8AwAAMB8EIxrwBc8QjAAAAJgeghEN+JERV1dh+wEA\nANAcIBjRoKiIyN6eqFUroXsCAADQ9CEY0QDLegEAAMwHwYgGWNYLAABgPghGNCgsRDACAABgLghG\nNMC+NAAAAOaDYKSGhw+JbtzAyAgAAIC5IBip4fp19hMjIwAAAOaBYKQGVF8FAAAwLwQjNfDVVzEy\nAgAAYB4IRmooKiJq04aofXuhewIAANA8IBipgV/WKxIJ3RMAAIDmAcFIDVjWCwAAYF4IRmpAwTMA\nAADzQjBSA0ZGAAAAzAvBiBqViqi4GCMjAAAA5oRgRE1pKdGjRxgZAQAAMCdBg5F169ZNdXNzu2pt\nbV3p5+eXmZ2d7VPX+Q8fPmw5f/78z9zc3K62atXqwbPPPntl+/bt4ern7N27d5S7u3uetbV1paen\nZ87+/fuHatsfFDwDAAAwP8GCkYSEhNEzZ878csmSJZ+cPHnSq2/fvqeDgoIOlJaWOtR2TUhIyPeH\nDx9+LS4ubsKFCxe679mzZ0yPHj3O86+np6f3Hzt27O7IyMgtp06dejE4ODgxODg48ezZs7206RNf\n8AzBCAAAgBlxHCfIo1+/flnTpk1bwz9XqVSijh07Fi5fvnyOpvP3798/xMbGpry8vNymtjZDQkIS\nhg8fvk/9mJ+fX8bkyZM3aDpfJpNJiIiTyWQcx3Hc+vUcZ2nJcY8fcwAAAFAHmUzG/e8zVMIZGBMI\nMjJSVVVlJZfLJQEBASn8MZFIxAUEBKRkZGT4a7pm3759I7y9vU8sX778406dOhX26NHj/OzZs1c+\nePCgFX9OZmamn3qbRERBQUEHamuzpsJCIhcXIgsLfd8ZAAAA6MpSiJuWlZXZK5VKCycnpxL1446O\njjfy8vLcNV1z+fLl544ePTrA2tq6MjExMbi0tNRhypQp62/evGkXFxc3gYhIoVA4a2pToVA419Wf\nGTNmkI2NDZ08SXT/PtGIEUShoaEUGhpq6FsFAABo9OLj4yk+Pr7asYqKCqO1L0gwog+VSiUWi8Wq\n77777t/t2rW7S0QUExMT9a9//es/GzZseK9ly5YP9W07NjaWJBIJBQYS2dgQ7d1rvH4DAAA0dpq+\noMvlcpJKpUZpX5BpGnt7+zILCwtlSUmJk/rxkpISJxcXl+uarnFxcbnu6upazAciRETu7u55HMeJ\nCgsLOxEROTs7K3Rps6bCQizrBQAAMDdBghErK6sqqVQqS0lJCeCPqVQqcWpq6mB/f/8MTdcMGDDg\naHFxsev9+/fb8McuXLjQXSwWqzp16lRIROTv75+h3iYR0cGDBwNra7OmoiKspAEAADA3wZb2RkVF\nxWzZsiVy165d43Jzcz3ee++9DZWVldbh4eHbiYjmzp37eVhY2E7+/LFjx+62s7O7GR4evj03N9fj\nyJEjL8+ePXtlRETENn6KZvr06V8lJycPiYmJicrLy3NfvHjxYrlcLnn//ffX1tefO3eI7t5FMAIA\nAGBuguWMhISEfF9aWuqwaNGiaIVC4ezl5XUyOTl5iIODQykRS0YtKCjozJ/fpk2b+wcPHgycNm3a\n197e3ifs7Oxujh49OmHp0qUL+HP8/f0zdu/ePXbBggVL582bt6x79+4XEhMTg3v27Hmuvv7wBc8w\nTQMAAGBeIu7/2rv3oCiufA/gpweBYUBQhAGGGBAMjKjA8MgAvrjIS69irBtdUhiJG+MTImpyY26V\nCG5V1BsVV2URr4WSWq+oSa2YjQs6KKKAyBuFgKhkQ3gMgxEUBhCZ3j+wdztEXCMNR+X7qfqVdJ8z\nPad/hdSvTnefZlnaY6CmpKTEw9PTs7i4uJjcu+dBgoMJuXOHEAcH2iMDAAB4uXE3sBYXF3t6eHiU\nDOVYeDfNE9zMiExGdxwAAACjDYqRJxoaCLGwIEQs/vd9AQAAQDgoRp7AY70AAAB0oBh5Ao/1AgAA\n0IFi5ImffkIxAgAAQAOKkScaGnCZBgAAgAYUI4SQR48IaWnBzAgAAAANKEYIIa2t/f9iZgQAAGDk\noRgh/bMihGBmBAAAgAYUIwTFCAAAAE0oRkh/MWJsTIiZGe2RAAAAjD4oRgghanX/rAjD0B4JAADA\n6INihBCi0eDmVQAAAFpQjJB/zYwAAADAyEMxQvrvGcHMCAAAAB0oRkj/OiOYGQEAAKADxQgh5PFj\nFCMAAAC0oBh5ApdpAAAA6EAx8gRmRgAAAOhAMUII0dMjRCqlPQoAAIDRCcUIIcTCor8gAQAAgJGH\nYoRgVgQAAIAmFCMExQgAAABNKEYIihEAAACaUIwQFCMAAAA0oRghKEYAAABoQjFCCLGyoj2C0eXE\niRO0hzDqIOcjDzkfecj5q4t6MZKYmLje3t7+ByMjoy4fH59rhYWF3oP1zc7O9heJRDp+6Onp9bW0\ntPxzbuPYsWMfDOwjkUi0zxqDq6uQZwT/Dv5gjDzkfOQh5yMPOX91jaH55SdPnvzd5s2b9yQnJ69W\nKpUFCQkJG0NCQjJramqcLS0tNYN9rra29q2xY8c+5LYH9jU1NX1w69YtJ26bYRj2WePAGiMAAAD0\nUJ0Z2bt376ZVq1YdjoyMTJXL5dWHDh1aI5FItCkpKb9/1ucsLCxapVJpCxcDiw2GYVh++7MKGwAA\nAKCLWjHy6NEjg5KSEo/AwEAVt49hGDYwMFCVn5/v+6zPuru7l8lkssbg4ODzeXl5fgPbOzo6TOzt\n7X948803f3znnXfOVFVVuQzHOQAAAMDQUbtM09raatHX16dnZWWl5u+XSqUt1dXV8qd9RiaTNSYn\nJ6/28vIq6u7uFh85cmSlv79/dkFBgVKhUJQSQohcLq8+evToCldX14q2trZxu3fv/sTPzy+vsrJy\nqq2tbQP/eN3d3WJCCFm5ciUxMTH5xXeFhoaS0NBQQc8Z+rW1tZGSkhLawxhVkPORh5yPPOR8+GRk\nZJCMjIxf7Ovo6CCEENLV1SUe8hewLEslGhoaZAzD6K5du6bk7//000//V6lUXnve48yZMyf7/fff\n/2qw9t7e3jGTJ0+u3bp16/aBbVevXvUjhLAIBAKBQCBeLHJzc/2GWhNQmxmxsLBo1dPT61Or1b94\nsFatVlvZ2Ng0Pe9xvL29C3Nzc2cM1j5mzJjHCoWi9Pbt25MHtikUirLi4mLP3zZyAAAA4Mjl8uqh\nHoNaMWJgYPDI09OzWKVSBYaFhZ0lhBCdTifKysqa+/HHH+9/3uOUlZW5y2SyxsHa+/r69CoqKlwX\nLFjw14FtEolE6+HhgTk9AAAAiqg+2rtp06a9kZGRqV5eXkXe3t6F+/bti+nq6jJasWLFUUII+fzz\nz3c0NjbKUlNTIwkhZN++fTEODg53XVxcqrh7RrKzs/3Pnz8fzB1z+/btsb6+vvmOjo532traxn35\n5Zef1tfXT1y5cuURWucJAAAAg6NajCxduvSURqOxjI2N3d7c3GytUChKMzIyQrlHcZubm63r6+sn\ncv17e3v1N2/evKehocFWIpFo3dzcylUqVeCcOXMuc33a2trGffTRR//X3NxsPX78+PteXl5FeXl5\nfkJMIwEAAIDwGJZlaY8BAAAARjHqy8HT9FuWooffJicnZ/bChQu/tbW1bRCJRLr09PRFA/vExsZu\nl8lkjRKJRBsUFHThaTcZw/PZsWPH597e3oWmpqYPrKys1IsXL/4LfxViDnIunKSkpLVubm7lZmZm\n7WZmZu1+fn55GRkZv1gPAPkeXjt37twiEol0GzduTODvR96FExcXFzfwFSsuLi5V/D5C5HvUFiPc\nUvTx8fHbSktLFW5ubuUhISGZGo3GkvbYXgdarVaiUChKExMT1xPy6yX5d+3a9dmBAweik5OTVxcU\nFCiNjY07Q0JCMnt6egzpjPjVlpOTMzs6OvpAQUGB8sKFC0G9vb36wcHB57VarYTrg5wLa+LEifW7\ndu36rKSkxKO4uNgzICDgYlhY2NnKysqphCDfw62wsND78OHDq1xdXSv4f1+Qd+FNmzbtZnNzszUX\nV69encm1CZZvWuuM0I633367IDo6ej+3rdPpGFtb25927tz5Ge2xvW7BMIwuPT09jJ9ra2vrpj17\n9mzi9rW3t5uKxeKutLS039Ee7+sQGo3GgmEY3ZUrV2Yi5yMX5ubm91JSUlYg38MbDx8+NHFycqrJ\nysoK8Pf3v7Rx48a9LIvf8+GIbdu2xbm7u5c+rU3IfI/KmZGhLEUPQ1dXVzdJrVZb8fNvamr6QKlU\nFiD/wmhraxtHCCHm5uY/E4KcD7e+vj69tLS08J6eHsNZs2ZdQb6H1/r16xMXLFjw14CAgIssyzLc\nfuR9eNTW1r5la2vb4OjoeGfZsmV/5h4sETLfVJ+moeVFlqIH4TQ3N1sTQsjA/FtZWam5NnhxOp1O\nFBMTs2/mzJlXuWu7yPnwuHHjxnRfX9/8np4eQyMjo65Tp04tnTx58m3unVnIt/DS0tLCy8rK3Ll7\n/PiXaPB7LjwfH59rqampkc7OzjWNjY2y+Pj4bbNmzbpy8+bNaULme1QWI/ByYlmWEYlEOtrjeNWt\nX78+saqqyoV/XXcwyPnQyOXy6oqKCtf29naz06dPLwkPD0/Lzs72H6w/8j009fX1Ezds2PBHlUoV\naGBg8IiQ/pzyZ0eeBnl/caGhof98Ic20adNuKpXKAjs7u7+fOnVq6WBLZrxIvkflZRqhlqKHF2Nt\nbd1MSH+++fvVarUV1wYvJioq6uC5c+fmX7p06T/4KxMj58NDX1+/18HB4a5CoSj94osv/kepVBYk\nJSWt5f6OIN/CKi4u9tRoNJYeHh4l+vr6vfr6+r05OTmz9+/f/7GBgcEj/J4PPzMzs3YnJ6dbd+7c\ncRTy93xUFiP8pei5fdxS9L6+vvk0xzYaTJo0qc7a2rqZn/8HDx6YXr9+/W3k/8WwLMtERUUdTE9P\nX3Tx4sUAOzu7v/PbkfOR0dfXp6fT6UTI9/AIDAxU3bx5c1p5eblbeXm5W1lZmbuXl1fRsmXL/lxW\nVuaOvA+/jo4Ok9ra2rdsbGyaBM037Tt1acXJkyeXisXirtTU1OVVVVVTVq1alWxubn6vpaXFkvbY\nXofo6OgwLi0tdS8tLXVnGEaXkJAQU1pa6v7jjz9OZFmW7Nq167/Hjx//89mzZxdWVFRMX7Ro0RlH\nR8fbPT09BrTH/irG2rVr/zRu3Lj7ly9fnt3U1GTNRVdXl5jrg5wLG1u2bNmRk5Mzq66uzr6iomL6\nli1bdujp6T3OysoKQL5HLubMmZMdExOTwG0j78LG5s2bd1++fHl2XV2dfW5url9gYOAFqVSqbm1t\nnSBkvqmfKM04ePDgejs7ux8MDQ27fXx88q9fv+5Ne0yvS1y6dMmfYRgdwzA6kUjUx/28YsWKFK5P\nbGxsvLW1dZNYLO4KCgo6X1tbO5n2uF/VGJhnLlJTU5fz+yHnwsWHH354xN7evs7Q0LBbKpWqg4KC\nzqtUqrnI98gG/9Fe5F34CA8PPyGTyRoMDQ2733jjjfr33nvv/+/evTtJ6HxjOXgAAACgalTeMwIA\nAAAvDxQjAAAAQBWKEQAAAKAKxQgAAABQhWIEAAAAqEIxAgAAAFShGAGA145IJNKdPXs2jPY4AOD5\noBgBAEF98MEHx0QikW5gzJ8//xztsQHAywlv7QUAQTEMw86bN+9vR48eXcHfb2ho2ENrTADwcsPM\nCAAIimVZxtDQsEcqlbbww8zMrJ2Q/ksohw4dWjNv3ry/SSQSraOj451vvvnmv/jHuHHjxvSAgICL\nEolEa2Fh0bp69erkzs5OY36flJSU30+dOrVSLBZ3y2Syxujo6AP8do1GY7l48eK/GBsbdzo5Od36\n9ttvFw7/2QPAi0AxAgCCY1mWeVb71q1b/7BkyZLTFRUVrhEREcfDw8PTqqur5YQQ0tnZaRwSEpI5\nYcKEe0VFRV6nT59eolKpAqOiog5yn09KSlobFRV1cM2aNYcqKyunfvfdd//p7Oxcw/+O+Pj4beHh\n4Wk3btyYPn/+/HMRERHH79+/P354zhgAhoT2S3gQCMTrFZGRkcfGjBnTa2Ji8pAfO3bs2MKy/S/1\nW7duXSL/Mz4+PvncvsOHD39kbm5+T6vVGnHt586dm6enp/eYe6u2TCZr2Lp16/bBxsAwjC42Njae\n2+7s7JQwDKPLzMwMpp0fBALx68A9IwAguICAgItJSUlr+fvMzc1/5n729fXN57f5+vrml5WVuRNC\nyPfffz/F3d29zMjIqItr9/Pzy9PpdKKamhpnlmWZpqYmm7lz52Y9awyurq4V3M8SiURramr6oKWl\nRTrUcwMA4aEYAQDBSSQSrYODw93n7c+yLMMwDMvfHqwvv0h5Fn19/V7+NsMwrE6nw6VpgJcQ/mMC\ngOD4hcXT5Ofn+/K3r1275jNlypTvCSHExcWlqry83E2r1Uq49tzc3BkikUjn7OxcM3bs2If29vY/\nqFSqwOEZPQCMNMyMAIDguru7xWq12oo/w6Gvr987YcKEe4QQ8vXXX7/r5eVVNGPGjNzjx49HFBUV\neXGPAkdERBzftm1bfGRkZGpcXFxcS0uLNDo6+sDy5cu/srS01BBCSFxcXNyaNWsOSaXSltDQ0IyH\nDx+OzcvL8+Pf5AoArw4UIwAgKIZh2IyMjFAbG5sm/n65XF5dVVXlQkj/ky5paWnh69at+5NMJms8\nceLEe3K5vJqQ/sswmZmZIRs2bPijt7d3oUQi0b777rtf7927dxN3rOXLl3/V3d0tTkhI2PjJJ5/s\ntrCwaF2yZMnpkT1TABAKw7LPnE0FABCUSCTSnTlz5p2wsLCztMcCAC8H3DMCAAAAVKEYAQAAAKpw\nmQYAAACowswIAAAAUIViBAAAAKhCMQIAAABUoRgBAAAAqlCMAAAAAFUoRgAAAIAqFCMAAABAFYoR\nAAAAoArFCAAAAFD1DxsKhnq6gATQAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x4929350>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "hist = solver.val_acc_history\n",
    "\n",
    "\"\"\"\n",
    "hist = [0.558, 0.67, 0.689, 0.709, 0.731, 0.767, 0.75, 0.799, 0.803, 0.816, \\\n",
    "        0.773, 0.797, 0.826, 0.831, 0.81, 0.823, 0.835, 0.834, 0.86, 0.825, \\\n",
    "        0.866, 0.861, 0.864, 0.87, 0.892, 0.831, 0.878, 0.866, 0.705, 0.881, \\\n",
    "        0.837, 0.893, 0.895, 0.893, 0.863, 0.884, 0.889, 0.91, 0.881, 0.898, \\\n",
    "        0.906, 0.908, 0.916, 0.915, 0.848, 0.813, 0.913, 0.927, 0.909, 0.906]\n",
    "\"\"\"\n",
    "plt.plot(hist)\n",
    "plt.xlabel(\"Epoch\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Possible assigments\n",
    "\n",
    "1. Instead of using vanilla RNN, try LSTM and GRU.\n",
    "2. Verify the effectiveness of data normalization.\n",
    "3. We can you conclude from the above learning curve figure, can to change learning rate to improve it?\n",
    "4. Add $l_2$ regularization term to the RNN model."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
